User interest-based enhancement of media quality

ABSTRACT

A mechanism is described for facilitating interest-based enhancement in media quality according to one embodiment. A method of embodiments, as described herein, includes receiving eye tracking information from a wearable computing device, where the eye tracking information contains data relating to pupils of eyes of a user having access to the wearable computing device. The method may further include enhancing quality of at least contents of media, if a size of a pupil is determined to have been altered due to the user&#39;s interest in the contents, where the media having the enhanced contents is encoded and communicated back to the wearable computing device to be rendered at the wearable computing device.

CLAIM TO PRIORITY

This Application is a continuation of and claims the benefit of andpriority to U.S. application Ser. No. 15/491,052, entitled USERINTEREST-BASED ENHANCEMENT OF MEDIA QUALITY, by Sean J. Lawrence, filedApr. 19, 2017, the entire contents of which are incorporated herein byreference.

FIELD

Embodiments described herein relate generally to data processing andmore particularly to facilitate user interest-based enhancement of mediaquality.

BACKGROUND

Conventional techniques are negatively influenced by environmentalchanges, such as ambient light, and thus, such conventional techniquesare not equipped to offer an intelligent rendering of media on displays.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are illustrated by way of example, and not by way oflimitation, in the figures of the accompanying drawings in which likereference numerals refer to similar elements.

FIG. 1 illustrates a computing device employing an interest-based mediaenhancement mechanism according to one embodiment.

FIG. 2 illustrates an interest-based media enhancement mechanismaccording to one embodiment.

FIG. 3A illustrates a system placement according to one embodiment.

FIG. 3B illustrates a transaction sequence using adjustment factors forfoveated media quality based on user's interest in content according toone embodiment.

FIG. 4A illustrates a method for preparing and rending enhanced mediabased on user interest according to one embodiment.

FIG. 4B illustrates a transaction sequence for determining region ofinterest on a sink device according to one embodiment.

FIG. 4C illustrates a transaction sequence for determining region ofinterest on a source device according to one embodiment.

FIG. 4D illustrates a transaction sequence using inputs to determinepsychophysical state according to one embodiment.

FIG. 4E illustrates a transaction sequence using inputs to determinepsychophysical state according to one embodiment.

FIG. 5 illustrates a computer device capable of supporting andimplementing one or more embodiments according to one embodiment.

FIG. 6 illustrates an embodiment of a computing environment capable ofsupporting and implementing one or more embodiments according to oneembodiment.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth.However, embodiments, as described herein, may be practiced withoutthese specific details. In other instances, well-known circuits,structures and techniques have not been shown in details in order not toobscure the understanding of this description.

Embodiments provide for a novel technique for facilitating qualityenhancement of one or more portions of media being rendered on a displaydevice based on a user's interest in contents of the one or moreportions such that the one or portions are regarded as a region ofinterest (ROI) or point of interest (POI). If the user lacks interest,then the media of normal quality is rendered.

In one embodiment, foveated rendering is based on where the user'svisual point of interest is located on the display where certain imagesare being rendered. This ROI may be determined by eye tracking, followedby detecting changes in the size of the user's pupils, where this ROImay be used to enhance the quality of the media contents within andaround the ROI being viewed by the user, such as by increasing bitrate.

FIG. 1 illustrates a computing device 100 employing an interest-basedmedia enhancement mechanism (“enhancement mechanism”) 110 according toone embodiment. Computing device 100 represents a communication and dataprocessing device including (but not limited to) smart wearable devices,smartphones, virtual reality (VR) devices, head-mounted display (HMDs),mobile computers, Internet of Things (IoT) devices, laptop computers,desktop computers, server computers, etc.

Computing device 100 may further include (without limitations) anautonomous machine or an artificially intelligent agent, such as amechanical agent or machine, an electronics agent or machine, a virtualagent or machine, an electro-mechanical agent or machine, etc. Examplesof autonomous machines or artificially intelligent agents may include(without limitation) robots, autonomous vehicles (e.g., self-drivingcars, self-flying planes, self-sailing boats, etc.), autonomousequipment (self-operating construction vehicles, self-operating medicalequipment, etc.), and/or the like. Throughout this document, “computingdevice” may be interchangeably referred to as “autonomous machine” or“artificially intelligent agent” or simply “robot”.

Computing device 100 may further include (without limitations) largecomputing systems, such as server computers, desktop computers, etc.,and may further include set-top boxes (e.g., Internet-based cabletelevision set-top boxes, etc.), global positioning system (GPS)-baseddevices, etc. Computing device 100 may include mobile computing devicesserving as communication devices, such as cellular phones includingsmartphones, personal digital assistants (PDAs), tablet computers,laptop computers, e-readers, smart televisions, television platforms,wearable devices (e.g., glasses, watches, bracelets, smartcards,jewelry, clothing items, etc.), media players, etc. For example, in oneembodiment, computing device 100 may include a mobile computing deviceemploying a computer platform hosting an integrated circuit (“IC”), suchas system on a chip (“SoC” or “SOC”), integrating various hardwareand/or software components of computing device 100 on a single chip.

As illustrated, in one embodiment, computing device 100 may include anynumber and type of hardware and/or software components, such as (withoutlimitation) graphics processing unit (“GPU” or simply “graphicsprocessor”) 114, graphics driver (also referred to as “GPU driver”,“graphics driver logic”, “driver logic”, user-mode driver (UMD), UMD,user-mode driver framework (UMDF), UMDF, or simply “driver”) 116,central processing unit (“CPU” or simply “application processor”) 112,memory 108, network devices, drivers, or the like, as well asinput/output (I/O) sources 104, such as touchscreens, touch panels,touch pads, virtual or regular keyboards, virtual or regular mice,ports, connectors, etc. Computing device 100 may include operatingsystem (OS) 106 serving as an interface between hardware and/or physicalresources of the computer device 100 and a user.

It is to be appreciated that a lesser or more equipped system than theexample described above may be preferred for certain implementations.Therefore, the configuration of computing device 100 may vary fromimplementation to implementation depending upon numerous factors, suchas price constraints, performance requirements, technologicalimprovements, or other circumstances.

Embodiments may be implemented as any or a combination of: one or moremicrochips or integrated circuits interconnected using a parentboard,hardwired logic, software stored by a memory device and executed by amicroprocessor, firmware, an application specific integrated circuit(ASIC), and/or a field programmable gate array (FPGA). The terms“logic”, “module”, “component”, “engine”, and “mechanism” may include,by way of example, software or hardware and/or combinations of softwareand hardware.

In one embodiment, enhancement mechanism 110 may be hosted orfacilitated by operating system 106 of computing device 100. In anotherembodiment, enhancement mechanism 110 may be hosted by or part ofgraphics processing unit (“GPU” or simply “graphics processor”) 114 orfirmware of graphics processor 114. Similarly, in yet anotherembodiment, enhancement mechanism 110 may be hosted by or part ofcentral processing unit (“CPU” or simply “application processor”) 112.In yet another embodiment, enhancement mechanism 110 may be hosted by orpart of any number and type of components of computing device 100, suchas a portion of enhancement mechanism 110 may be hosted by or part ofoperating system 106, another portion may be hosted by or part ofgraphics processor 114, another portion may be hosted by or part ofapplication processor 112, while one or more portions of enhancementmechanism 110 may be hosted by or part of operating system 106 and/orany number and type of devices of computing device 100. It iscontemplated that one or more portions or components of enhancementmechanism 110 may be employed as hardware, software, and/or firmware.

It is contemplated that embodiments are not limited to any particularimplementation or hosting of enhancement mechanism 110 and thatenhancement mechanism 110 and one or more of its components may beimplemented as hardware, software, firmware, or any combination thereof.

Computing device 100 may host network interface(s) to provide access toa network, such as a LAN, a wide area network (WAN), a metropolitan areanetwork (MAN), a personal area network (PAN), Bluetooth, a cloudnetwork, a mobile network (e.g., 3^(rd) Generation (3G), 4^(th)Generation (4G), etc.), an intranet, the Internet, etc. Networkinterface(s) may include, for example, a wireless network interfacehaving antenna, which may represent one or more antenna(e). Networkinterface(s) may also include, for example, a wired network interface tocommunicate with remote devices via network cable, which may be, forexample, an Ethernet cable, a coaxial cable, a fiber optic cable, aserial cable, or a parallel cable.

Embodiments may be provided, for example, as a computer program productwhich may include one or more machine-readable media having storedthereon machine-executable instructions that, when executed by one ormore machines such as a computer, network of computers, or otherelectronic devices, may result in the one or more machines carrying outoperations in accordance with embodiments described herein. Amachine-readable medium may include, but is not limited to, floppydiskettes, optical disks, CD-ROMs (Compact Disc-Read Only Memories), andmagneto-optical disks, ROMs, RAMs, EPROMs (Erasable Programmable ReadOnly Memories), EEPROMs (Electrically Erasable Programmable Read OnlyMemories), magnetic or optical cards, flash memory, or other type ofmedia/machine-readable medium suitable for storing machine-executableinstructions.

Moreover, embodiments may be downloaded as a computer program product,wherein the program may be transferred from a remote computer (e.g., aserver) to a requesting computer (e.g., a client) by way of one or moredata signals embodied in and/or modulated by a carrier wave or otherpropagation medium via a communication link (e.g., a modem and/ornetwork connection).

Throughout the document, term “user” may be interchangeably referred toas “viewer”, “observer”, “person”, “individual”, “end-user”, and/or thelike. It is to be noted that throughout this document, terms like“graphics domain” may be referenced interchangeably with “graphicsprocessing unit”, “graphics processor”, or simply “GPU” and similarly,“CPU domain” or “host domain” may be referenced interchangeably with“computer processing unit”, “application processor”, or simply “CPU”.

It is to be noted that terms like “node”, “computing node”, “server”,“server device”, “cloud computer”, “cloud server”, “cloud servercomputer”, “machine”, “host machine”, “device”, “computing device”,“computer”, “computing system”, and the like, may be usedinterchangeably throughout this document. It is to be further noted thatterms like “application”, “software application”, “program”, “softwareprogram”, “package”, “software package”, and the like, may be usedinterchangeably throughout this document. Also, terms like “job”,“input”, “request”, “message”, and the like, may be used interchangeablythroughout this document.

FIG. 2 illustrates enhancement mechanism 110 of FIG. 1 according to oneembodiment. For brevity, many of the details already discussed withreference to FIG. 1 are not repeated or discussed hereafter. In oneembodiment, enhancement mechanism 110 may include any number and type ofcomponents, such as (without limitations): detection/reception logic201; evaluation logic 203; encoding/enhancement logic 205; andcommunication/compatibility logic 207.

Computing device 100 is further shown to include user interface 219(e.g., graphical user interface (GUI)-based user interface, Web browser,cloud-based platform user interface, software application-based userinterface, other user or application programming interfaces (APIs)etc.). Computing device 100 may further include I/O source(s) 108 havingcapturing/sensing component(s) 231, such as camera(s) 241 (e.g., Intel®RealSense™ camera), and output component(s) 233, such as display(s) 243(e.g., integral displays, tensor displays, etc.).

Computing device 100 is further illustrated as having access to and/orbeing in communication with one or more database(s) 225 and/or one ormore of other computing devices over one or more communication medium(s)230 (e.g., networks such as a cloud network, a proximity network, theInternet, etc.).

In some embodiments, database(s) 225 may include one or more of storagemediums or devices, repositories, data sources, etc., having any amountand type of information, such as data, metadata, etc., relating to anynumber and type of applications, such as data and/or metadata relatingto one or more users, physical locations or areas, applicable laws,policies and/or regulations, user preferences and/or profiles, securityand/or authentication data, historical and/or preferred details, and/orthe like.

As aforementioned, computing device 100 may host I/O sources 108including capturing/sensing component(s) 231 and output component(s)233. In one embodiment, capturing/sensing component(s) 231 may includesensor array (such as microphones or microphone array (e.g., ultrasoundmicrophones), cameras or camera array (e.g., two-dimensional (2D)cameras, three-dimensional (3D) cameras, infrared (IR) cameras,depth-sensing cameras, etc.), capacitors, radio components, radarcomponents, etc.), scanners, accelerometers, etc. Similarly, outputcomponent(s) 233 may include any number and type of display devices orscreens, projectors, speakers, light-emitting diodes (LEDs), one or morespeakers and/or vibration motors, etc.

As illustrated, computing device 100 may be in communication withanother computing device 250, where computing device 100 may include andhereinafter referred to as a server computer, while computing device 250may include and hereafter referred to as an HMD. In one embodiment,server computer 100 may host enhancement mechanism 110, while HMD 250may host eye tracking and media rendering mechanism (“tracking/renderingmechanism”) 260. In another embodiment, HDM 250 may host bothenhancement mechanism 110 and tracking/rendering mechanism 260. In yetanother embodiment, HMD 250 may host or have access to both enhancementmechanism 110 and tracking/rendering mechanism 260, but depending on theavailability and extent of the system resources at HMD 250, tasks may bealternated between HMD 250 and server computer 100. For example, smalltasks that are not overly power-consuming, memory-consuming, etc., maybe performed at HMD 250, while larger tasks requiring greater amount ofsystem resources may be performed at server computer 100.

In the illustrated embodiment, HMD 250 is shown as hostingtracking/rendering mechanism 260 including any number and type ofcomponents, such as (without limitations): tracking logic 261;decoding/rendering logic 263; and communication logic 265. HMD 250 isfurther shown as having display screen 267 for rendering and displayingof contents, such as still images, videos, etc., and one or moresensors/trackers 269 for sensing, scanning, or tracking a number ofthings, such as the user's eyes, pupils, iris, etc. It is contemplatedthat HMD 250 may contain any number and type of other computing andprocessing components, such as I/O sources, user interface, etc., butfor the sake of brevity, such details are contemplated and not shown.

For example, in one embodiment, tracking/reception logic 261 may be usedto track the size of the user's pupil along with other factors that arecapable of influencing pupil size. It is contemplated that the user mayrefer to a person having access to and using or wearing HMD 250. In oneembodiment, this tracked information may then be collected andcommunicated over to enhancement mechanism 110 at server computer 100over one or more communication mediums 230 (e.g., cloud network,Internet, proximity network, etc.) as facilitated by communication logic265 and communication/compatibility logic 207.

At enhancement mechanism 110, detection/reception logic 201 may be usedto receive this collection of tracked information and forwards it on toevaluation logic 203 for further processing and analysis, such as fordetection ROI and determining whether the user's interest in ROI is dueto the contents being rendered as part of ROI or for other reasons (suchas changes in surrounding lights, weather, noise, movement, etc.). If,for example, it is determined that the user's pupil size is dilatedbecause of the user being interested in the contents of the ROI, thenencoding/enhancement logic 205 may be used to increase or enhance thequality of the media being delivered as part of the ROI to provide theuser with a better experience of contents at least within the ROI.

This enhanced version of the media may then be encoded byencoding/enhancement logic 205 and forwarded on to HMD 250 overcommunication medium(s) 230 (e.g., cloud network, Internet, proximitynetwork, etc.) as facilitated by communication/compatibility logic 207and communication logic 265. In one embodiment, tracking/reception logic261 may receive this enhanced version of the media and forwards it on todecoding/rendering logic 263 to then decode the media and render it ondisplay screen 267 for the user to view and experience.

It is contemplated that pupil size or a translated size of a pupil atcertain distances (e.g., 1 meter) from HMD 250 or the relative size ofthe pupil with respect to the iris of the eye may be measured using oneor more sensors/trackers 269 (e.g., cameras, eye trackers, irisscanners, movement/size sensors, etc.). It is contemplated that pupilsize is linked to illumination on the eyes of the user, such as due tovisual stimuli as well as other forms of emotional and physical arousaland excitement, such as happiness, anxiety, love, touch, etc., can havean impact on the size of the pupil. For example, a sudden turning on ofthe light in the room or the touch of a loved one can expand the size ofthe pupil. Now, since this can happened for any number of reasons whilesuing HMD 250, it is therefore determined that any changes in the sizeof the user's pupil is due to the user's interest in the contents withinthe ROI and not due to some external influences, such as lighting,touching, etc.

While visual contents can have significant influence on the pupil size,but to ensure that the change in the pupil size is an indication of theuser's interest in certain visual contents, evaluation logic 203 is usedto evaluate and perform this distinction as to whether the user's pupilsize was changed due to the user's interest in contents or otherinfluences evaluation logic 203. For example, evaluation logic 203 maydetect, monitor, and evaluate certain indicators that are representativeof other influences in order to determine whether the change in thepupil size is due to the user's interest in the contents.

It is to be noted and contemplated that this novel technique is notlimited to merely any particular form of media ant that it isapplication to and implemented with all forms of media, such as stillimages, audio, video, etc. Embodiments provide for a novel technique topropose an increase in video and audio bitrate when the user findscertain media contents of interest by detecting instances of pupildilation due to actual visual stimulation and not simply due to otherinstances as listed above. For example, when the pupils are not dilated,the bitrate and the corresponding image quality may be set to nominalvalues as in a normal foveated rendering of contents.

In one embodiment, this novel technique may be applied to tethered (suchas wired) or untethered (such as wireless) HMDs, such as HMD 250. Asdescribed earlier, in some embodiments, tracking/rendering mechanism 260hosted by HMD 250 may be responsible for performing eye and pupil sizetracking using one or more of sensors/trackers 269 as facilitated bytracking/reception logic 261. Any information obtained from thistracking may then be communicated or fed back to enhancement mechanism110 at server computer 100, which triggers evaluation logic 203 toevaluate the information to look for parameters or other clues todetermine whether the change in pupil size was due to interest incontents or other factors playing a role.

Based on its findings and if the pupil size is determined to have beenchanged due to interest in contents, evaluation logic 203 then instructsencoding/enhancement logic 205 to create an ROI based on the user'sgaze/foveated region and adjust the quality of the ROI based on theusers' pupil size, etc. Once the ROI has been determined and created,encoding/enhancement logic 205 may then concede the video frame and sendthis data to HMD 250 through communication/compatibility logic 207 overcommunication medium(s) 230.

Once received by tracking/reception logic 261 at HMD 250, this encodedand enhanced media is then decoded by decoding/rendering logic 263 andrendered using display screen 267. In one embodiment, this communicationbetween server computer 100 and HMD 250 may be continuous and non-stop;while, in another embodiment, it is done over regular intervals orperiods of time; while, in yet another embodiment, it is done uponoccurrence of an event, such as updates to software, etc.; while, in yetanother embodiment, it done on-demand Further, this cycle of creatingand encoding of ROI may be repeated based on the new eye data, where afoveated frame may be correspondingly created and sent over to HMD 250for decoding and rendering.

For example, in some embodiments, once the user's interest in certaincontents is identified, appropriate bitrate adjustments (or otherquality parameters) may be made to improve the quality of what isrendered in the foveated region. In some embodiments, other variationsof image quality improvements may be made as described earlier. Theextent of improvements or reduction in quality may be derived frommachine learning-based methods for the particular user with respect tocontent affinity and ratings over time. Moreover, user variations inpupil size to brightness relationship may be determined by the system byvarious learning algorithms and user preferences.

It is contemplated that increasing the quality of video, images, andaudio, in response to the user's interest in contents is likely toincrease the user's interest in those contents and would lead to animproved experience for the user. For example, during these instancesand apart from increasing bitrate of foveated regions as describedabove, in one embodiment, it is contemplated that the entire media'squality may be improved and not just the quality of the ROI. In somecases, along with the quality of ROI, the quality of other regions, suchas those surrounding the ROI, may also be enhanced, offering the user ahigh level of media experience.

For example, in some cases, foveated rendering may be switched on or offif the user is particularly interested in the content being rendered.Further, when applied to normal foveated rendering, the bitrate betweenthe foveated region and other regions may be increased, while retainingthe overall bandwidth requirements the same, but improving the qualityof the region that interests the user. For example, when applied inwired systems, benefits for reducing compute and power may be taken intoconsideration to take advantage of various foveated/ROI schemes. Asaforementioned, this novel technique may also be applied to computingsystems, such as HMD 250, where all or most of processing and foveatedregion creation, etc., may be performed at HMD 250 instead of at otherdevices, such as server computer 100, where associated benefits may bein reduced computing and power consumption may be achieved.

Capturing/sensing component(s) 231 may further include one or more ofvibration components, tactile components, conductance elements,biometric sensors, chemical detectors, signal detectors,electroencephalography, functional near-infrared spectroscopy, wavedetectors, force sensors (e.g., accelerometers), illuminators,eye-tracking or gaze-tracking system, head-tracking system, etc., thatmay be used for capturing any amount and type of visual data, such asimages (e.g., photos, videos, movies, audio/video streams, etc.), andnon-visual data, such as audio streams or signals (e.g., sound, noise,vibration, ultrasound, etc.), radio waves (e.g., wireless signals, suchas wireless signals having data, metadata, signs, etc.), chemicalchanges or properties (e.g., humidity, body temperature, etc.),biometric readings (e.g., figure prints, etc.), brainwaves, braincirculation, environmental/weather conditions, maps, etc. It iscontemplated that “sensor” and “detector” may be referencedinterchangeably throughout this document. It is further contemplatedthat one or more capturing/sensing component(s) 231 may further includeone or more of supporting or supplemental devices for capturing and/orsensing of data, such as illuminators (e.g., IR illuminator), lightfixtures, generators, sound blockers, etc.

It is further contemplated that in one embodiment, capturing/sensingcomponent(s) 231 may further include any number and type of contextsensors (e.g., linear accelerometer) for sensing or detecting any numberand type of contexts (e.g., estimating horizon, linear acceleration,etc., relating to a mobile computing device, etc.). For example,capturing/sensing component(s) 231 may include any number and type ofsensors, such as (without limitations): accelerometers (e.g., linearaccelerometer to measure linear acceleration, etc.); inertial devices(e.g., inertial accelerometers, inertial gyroscopes,micro-electro-mechanical systems (MEMS) gyroscopes, inertial navigators,etc.); and gravity gradiometers to study and measure variations ingravitation acceleration due to gravity, etc.

Further, for example, capturing/sensing component(s) 231 may include(without limitations): audio/visual devices (e.g., cameras, microphones,speakers, etc.); context-aware sensors (e.g., temperature sensors,facial expression and feature measurement sensors working with one ormore cameras of audio/visual devices, environment sensors (such as tosense background colors, lights, etc.); biometric sensors (such as todetect fingerprints, etc.), calendar maintenance and reading device),etc.; global positioning system (GPS) sensors; resource requestor;and/or TEE logic. TEE logic may be employed separately or be part ofresource requestor and/or an I/O subsystem, etc. Capturing/sensingcomponent(s) 231 may further include voice recognition devices, photorecognition devices, facial and other body recognition components,voice-to-text conversion components, etc.

Similarly, output component(s) 233 may include dynamic tactile touchscreens having tactile effectors as an example of presentingvisualization of touch, where an embodiment of such may be ultrasonicgenerators that can send signals in space which, when reaching, forexample, human fingers can cause tactile sensation or like feeling onthe fingers. Further, for example and in one embodiment, outputcomponent(s) 233 may include (without limitation) one or more of lightsources, display devices and/or screens, audio speakers, tactilecomponents, conductance elements, bone conducting speakers, olfactory orsmell visual and/or non/visual presentation devices, haptic or touchvisual and/or non-visual presentation devices, animation displaydevices, biometric display devices, X-ray display devices,high-resolution displays, high-dynamic range displays, multi-viewdisplays, and head-mounted displays (HMDs) for at least one of virtualreality (VR) and augmented reality (AR), etc.

It is contemplated that embodiment are not limited to any particularnumber or type of use-case scenarios, architectural placements, orcomponent setups; however, for the sake of brevity and clarity,illustrations and descriptions are offered and discussed throughout thisdocument for exemplary purposes but that embodiments are not limited assuch. Further, throughout this document, “user” may refer to someonehaving access to one or more computing devices, such as computing device100, and may be referenced interchangeably with “person”, “individual”,“human”, “him”, “her”, “child”, “adult”, “viewer”, “player”, “gamer”,“developer”, “programmer”, and/or the like.

Communication/compatibility logic 207 may be used to facilitate dynamiccommunication and compatibility between various components, networks,computing devices, database(s) 225, and/or communication medium(s) 230,etc., and any number and type of other computing devices (such aswearable computing devices, mobile computing devices, desktop computers,server computing devices, etc.), processing devices (e.g., centralprocessing unit (CPU), graphics processing unit (GPU), etc.),capturing/sensing components (e.g., non-visual data sensors/detectors,such as audio sensors, olfactory sensors, haptic sensors, signalsensors, vibration sensors, chemicals detectors, radio wave detectors,force sensors, weather/temperature sensors, body/biometric sensors,scanners, etc., and visual data sensors/detectors, such as cameras,etc.), user/context-awareness components and/oridentification/verification sensors/devices (such as biometricsensors/detectors, scanners, etc.), memory or storage devices, datasources, and/or database(s) (such as data storage devices, hard drives,solid-state drives, hard disks, memory cards or devices, memorycircuits, etc.), network(s) (e.g., Cloud network, Internet, Internet ofThings, intranet, cellular network, proximity networks, such asBluetooth, Bluetooth low energy (BLE), Bluetooth Smart, Wi-Fi proximity,Radio Frequency Identification, Near Field Communication, Body AreaNetwork, etc.), wireless or wired communications and relevant protocols(e.g., Wi-Fi®, WiMAX, Ethernet, etc.), connectivity and locationmanagement techniques, software applications/websites, (e.g., socialand/or business networking websites, business applications, games andother entertainment applications, etc.), programming languages, etc.,while ensuring compatibility with changing technologies, parameters,protocols, standards, etc.

Throughout this document, terms like “logic”, “component”, “module”,“framework”, “engine”, “tool”, and/or the like, may be referencedinterchangeably and include, by way of example, software, hardware,and/or any combination of software and hardware, such as firmware. Inone example, “logic” may refer to or include a software component thatis capable of working with one or more of an operating system, agraphics driver, etc., of a computing device, such as computing devices100, 250. In another example, “logic” may refer to or include a hardwarecomponent that is capable of being physically installed along with or aspart of one or more system hardware elements, such as an applicationprocessor, a graphics processor, etc., of a computing device, such ascomputing devices 100, 250. In yet another embodiment, “logic” may referto or include a firmware component that is capable of being part ofsystem firmware, such as firmware of an application processor or agraphics processor, etc., of a computing device, such as computingdevices 100, 250.

Further, any use of a particular brand, word, term, phrase, name, and/oracronym, such as “eye tracking”, “iris scanning”, “pupil tracking”,“pupil size”, “interest in content”, “region of interest”, “ROI”,“encoding”, “decoding”, “enhancement”, “rendering”, “automatic”,“dynamic”, “user interface”, “camera”, “sensor”, “microphone”, “displayscreen”, “speaker”, “verification”, “authentication”, “privacy”, “user”,“user profile”, “user preference”, “sender”, “receiver”, “personaldevice”, “smart device”, “mobile computer”, “wearable device”, “IoTdevice”, “proximity network”, “cloud network”, “server computer”, etc.,should not be read to limit embodiments to software or devices thatcarry that label in products or in literature external to this document.

It is contemplated that any number and type of components may be addedto and/or removed from enhancement mechanism 110 and/ortracking/rendering mechanism 260 to facilitate various embodimentsincluding adding, removing, and/or enhancing certain features. Forbrevity, clarity, and ease of understanding of enhancement mechanism 110and/or tracking/rendering mechanism 260, many of the standard and/orknown components, such as those of a computing device, are not shown ordiscussed here. It is contemplated that embodiments, as describedherein, are not limited to any particular technology, topology, system,architecture, and/or standard and are dynamic enough to adopt and adaptto any future changes.

FIG. 3A illustrates a system placement 300 according to one embodiment.For brevity, many of the details previously discussed with reference toFIGS. 1-2 may not be discussed or repeated hereafter. Any processesrelating to system placement 300 may be performed by processing logicthat may comprise hardware (e.g., circuitry, dedicated logic,programmable logic, etc.), software (such as instructions run on aprocessing device), or a combination thereof, as facilitated byenhancement mechanism 110 and/or tracking/rendering mechanism 260 ofFIG. 2. The processes associated with system placement 300 may beillustrated or recited in linear sequences for brevity and clarity inpresentation; however, it is contemplated that any number of them can beperformed in parallel, asynchronously, or in different orders. Further,embodiments are not limited to any particular architectural placement,framework, transaction sequence, and/or structure of components and/orprocesses, such as system placement 300.

As described with reference to FIG. 2, system placement 300 employs acouple of computing devices, such as a server computer as source 100 andan HMD as sink 250. As illustrated, source 100 pay perform one or moretasks like creating ROI, modifying ROI quality, encoding, transmittingenhanced media, and/or the like, using host enhancement mechanism 100while communicating encoded foveated media (e.g., video stream) to sink250 through forward channel 301 over a communication network (e.g.,Internet, Bluetooth, proximity, etc.). Similarly, sink 250 performsdecoding and rendering of the media, tracking of the user's eyes, pupilsize, etc., using tracking/rendering mechanism 260, while communicatingeye tracking coordinates, pupil dilation, etc., to source 100 throughreverse channel 303 over a communication network.

FIG. 3B illustrates a transaction sequence 350 using adjustment factorsfor foveated media quality based on user interest in contents accordingto one embodiment. For brevity, many of the details previously discussedwith reference to FIGS. 1-3A may not be discussed or repeated hereafter.Any processes relating to transaction sequence 350 may be performed byprocessing logic that may comprise hardware (e.g., circuitry, dedicatedlogic, programmable logic, etc.), software (such as instructions run ona processing device), or a combination thereof, as facilitated byenhancement mechanism 110 and/or tracking/rendering mechanism 260 ofFIG. 2. The processes associated with transaction sequence 350 may beillustrated or recited in linear sequences for brevity and clarity inpresentation; however, it is contemplated that any number of them can beperformed in parallel, asynchronously, or in different orders.

In the illustrated embodiment, adjustment factors may include (withoutlimitations) display brightness 351, visual context 353, aural context355, biometrics 357, and device interaction context 359, and/or thelike, that are then used to prepare and consider user psychophysicalstate 361 (such as whether pupil dilation is due to interest incontents). As illustrated, audio-visual (AV) media quality bitrate,etc., may then be adjusted based on user psychophysical state 361.

FIG. 4A illustrates a method 400 for preparing and rendering enhancedmedia based on user interest according to one embodiment. For brevity,many of the details previously discussed with reference to FIGS. 1-3Bmay not be discussed or repeated hereafter. Any processes relating tomethod 400 and transaction sequences 420, 450, 460 and 480 of FIGS. 4B,4C, 4D and 4E, respectively, may be performed by processing logic thatmay comprise hardware (e.g., circuitry, dedicated logic, programmablelogic, etc.), software (such as instructions run on a processingdevice), or a combination thereof, as facilitated by enhancementmechanism 110 and/or tracking/rendering mechanism 260 of FIG. 2. Theprocesses associated with method 400 and transaction sequences 420, 450,460, 480 may be illustrated or recited in linear sequences for brevityand clarity in presentation; however, it is contemplated that any numberof them can be performed in parallel, asynchronously, or in differentorders.

Method 400 begins at block 401 with tracking of eyes, pupils, etc., at awearable computing device, such as HMD, where this tracking informationis collected and transmitted over to another computing device, such asserver computer over a communication network, etc. This trackinginformation is received at block 403 and evaluates to determine whetherthere was any change in the size of the pupil of the user of the HMD atblock 405. If not, at block 407, media (e.g., images, videos, etc.) ofnormal quality is communicated form the server computer to the HMD forrendering of this media at the HMD using one or more display screens.If, however, there has been a change in the pupil size of the user,then, at block 409, another determination is made as to whether thischange is due to user interest in certain contents of the media orbecause of changes in or occurrence of other factors, such as light,environment, weather, human touch, accident, horror, excitement, etc.,surrounding the user.

If the pupil size did not change due to the user's interest in contents,then method 400 continues at block 407. If the changes were due to theuser's interest in contents, then, in one embodiment, an ROI is createdencompassing the contents of interest to the user and then, at the ROIportion of the media is adjusted for quality of media, such that thequality of the contents of the ROI (or portions of the media surroundingthe ROI or to the whole media) is enhance for superior user experienceat the HMD at block 411. At block 413, the enhanced media is encoded andthe encoded enhanced media is sent over to the HMD over a communicationnetwork. At block 415, the HMD receives the encoded enhanced media,decodes it, and renders the contents of the media on a display screen ofthe HMD for viewing by the user.

FIG. 4B illustrates a transaction sequence 420 for determining ROI on asink device 250 according to one embodiment. For brevity, many of thedetails previously discussed with reference to FIGS. 1-4A may not bediscussed or repeated hereafter. As illustrated here, sink device 250may include a wearable computing device, such as an HMD, while a sourcedevice 100 may include any computing device, such as a server computer,a desktop computer, a mobile device, etc. In the illustrated embodiment,at sink device 250, a user's psychological state is determined if thechange in the size of the user's pupil is due the user's interest in theaudio-visual (AV) contents at 431. In one embodiment, this is determinedthrough eye and pupil size tracking at 433 and computing the ROI and ROIquality/bitrate at 435.

In one embodiment, reverse channel 427 provides for ROI coordinates andROI quality and bitrate to be communicated on to source device 100,where ROI quality is modified at 421, encoded at 423, and subsequently,transmitted 425 on to sink device 250 through forward channel 429 tocarry the encoded foveated video stream over one or more networks, suchas a cloud network, a proximity network, and the Internet. At sinkdevice 250, the encoded video stream is received at 437, decoded at 439,and subsequently, the contents of the video stream are rendered at 441using a display screen.

FIG. 4C illustrates a transaction sequence 450 for determining ROI on asource device 100 according to one embodiment. For brevity, many of thedetails previously discussed with reference to FIGS. 1-4B may not bediscussed or repeated hereafter. As illustrated here, at sink device250, a user's psychological state is determined if the change in thesize of the user's pupil is due the user's interest in AV content at431, where this is determined through eye and pupil size tracking at433.

In one embodiment, reverse channel 453 provides for communication eyetracking coordinates, pupil dilation, etc., on to source device 100,where ROI is determined at 451, ROI quality is modified at 421, encodedat 423, and subsequently, transmitted 425 on to sink device 100 througha forward channel 455 to carry the encoded foveated video stream overone or more networks, such as a cloud network, a proximity network, andthe Internet. At sink device 250, the encoded video stream is receivedat 437, decoded at 439, and subsequently, the contents of the videostream are rendered at 441 using a display screen.

FIG. 4D illustrates a transaction sequence 460 using inputs to determinepsychophysical state according to one embodiment. For brevity, many ofthe details previously discussed with reference to FIGS. 1-4C may not bediscussed or repeated hereafter. As illustrated here, at source device100, partial inputs to determine user psychophysical state are gatheredat 461 and using supplemental forward channel 463, these partial inputsare communicated over to sink device 250, where partial input 469 areused to determine if pupil change is due to the user's interest in AVcontents at 471. Further, at sink device 250, eye and pupil sizetracking is used at 433 to determine ROI and ROI quality and bitrate at435.

In one embodiment, over reverse channel 465, ROI coordinates and qualityand bitrates are communicated over to source device 100 where ROIquality is modified at 421, encoded at 423, and transmitted at 425 overto sink device 250. At sink device 250, the encoded foveated videostream is received at 437 using forward channel 467, where this encodedvideo stream is decoded at 439, and then rendered at 441 using a displayscreen.

FIG. 4E illustrates a transaction sequence 480 using inputs to determinepsychophysical state according to one embodiment. For brevity, many ofthe details previously discussed with reference to FIGS. 1-4D may not bediscussed or repeated hereafter. As illustrated here, at source device100, partial inputs to determine user psychophysical state are gatheredat 461 and using supplemental forward channel 481, these partial inputsare communicated over to sink device 250, where partial input 469 areused to determine if pupil change is due to the user's interest in AVcontents at 471. Further, at sink device 250, eye and pupil sizetracking is performed at 433 and communicated over to source device 100using reverse channel 483, where, at source device 100, ROI and itsquality and bitrate are determined at 435.

This is followed by modifying of ROI quality at 421, encoding of ROI at423, and transmitting at 425 of the encoded foveated video stream overto sink device 250 using forward channel 485. At sink device 250, theencoded video stream is received at 437, decoded 429, and rendered at441 using a display screen.

FIG. 5 illustrates a computing device 500 in accordance with oneimplementation. The illustrated computing device 500 may be same as orsimilar to computing device 100 of FIG. 1. The computing device 500houses a system board 502. The board 502 may include a number ofcomponents, including but not limited to a processor 504 and at leastone communication package 506. The communication package is coupled toone or more antennas 516. The processor 504 is physically andelectrically coupled to the board 502.

Depending on its applications, computing device 500 may include othercomponents that may or may not be physically and electrically coupled tothe board 502. These other components include, but are not limited to,volatile memory (e.g., DRAM) 508, non-volatile memory (e.g., ROM) 509,flash memory (not shown), a graphics processor 512, a digital signalprocessor (not shown), a crypto processor (not shown), a chipset 514, anantenna 516, a display 518 such as a touchscreen display, a touchscreencontroller 520, a battery 522, an audio codec (not shown), a video codec(not shown), a power amplifier 524, a global positioning system (GPS)device 526, a compass 528, an accelerometer (not shown), a gyroscope(not shown), a speaker 530, cameras 532, a microphone array 534, and amass storage device (such as hard disk drive) 510, compact disk (CD)(not shown), digital versatile disk (DVD) (not shown), and so forth).These components may be connected to the system board 502, mounted tothe system board, or combined with any of the other components.

The communication package 506 enables wireless and/or wiredcommunications for the transfer of data to and from the computing device500. The term “wireless” and its derivatives may be used to describecircuits, devices, systems, methods, techniques, communicationschannels, etc., that may communicate data through the use of modulatedelectromagnetic radiation through a non-solid medium. The term does notimply that the associated devices do not contain any wires, although insome embodiments they might not. The communication package 506 mayimplement any of a number of wireless or wired standards or protocols,including but not limited to Wi-Fi (IEEE 802.11 family), WiMAX (IEEE802.16 family), IEEE 802.20, long term evolution (LTE), Ev-DO, HSPA+,HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, DECT, Bluetooth, Ethernetderivatives thereof, as well as any other wireless and wired protocolsthat are designated as 3G, 4G, 5G, and beyond. The computing device 500may include a plurality of communication packages 506. For instance, afirst communication package 506 may be dedicated to shorter rangewireless communications such as Wi-Fi and Bluetooth and a secondcommunication package 506 may be dedicated to longer range wirelesscommunications such as GPS, EDGE, GPRS, CDMA, WiMAX, LTE, Ev-DO, andothers.

The cameras 532 including any depth sensors or proximity sensor arecoupled to an optional image processor 536 to perform conversions,analysis, noise reduction, comparisons, depth or distance analysis,image understanding and other processes as described herein. Theprocessor 504 is coupled to the image processor to drive the processwith interrupts, set parameters, and control operations of imageprocessor and the cameras. Image processing may instead be performed inthe processor 504, the graphics CPU 512, the cameras 532, or in anyother device.

In various implementations, the computing device 500 may be a laptop, anetbook, a notebook, an ultrabook, a smartphone, a tablet, a personaldigital assistant (PDA), an ultra mobile PC, a mobile phone, a desktopcomputer, a server, a set-top box, an entertainment control unit, adigital camera, a portable music player, or a digital video recorder.The computing device may be fixed, portable, or wearable. In furtherimplementations, the computing device 500 may be any other electronicdevice that processes data or records data for processing elsewhere.

Embodiments may be implemented using one or more memory chips,controllers, CPUs (Central Processing Unit), microchips or integratedcircuits interconnected using a motherboard, an application specificintegrated circuit (ASIC), and/or a field programmable gate array(FPGA). The term “logic” may include, by way of example, software orhardware and/or combinations of software and hardware.

References to “one embodiment”, “an embodiment”, “example embodiment”,“various embodiments”, etc., indicate that the embodiment(s) sodescribed may include particular features, structures, orcharacteristics, but not every embodiment necessarily includes theparticular features, structures, or characteristics. Further, someembodiments may have some, all, or none of the features described forother embodiments.

In the following description and claims, the term “coupled” along withits derivatives, may be used. “Coupled” is used to indicate that two ormore elements co-operate or interact with each other, but they may ormay not have intervening physical or electrical components between them.

As used in the claims, unless otherwise specified, the use of theordinal adjectives “first”, “second”, “third”, etc., to describe acommon element, merely indicate that different instances of likeelements are being referred to, and are not intended to imply that theelements so described must be in a given sequence, either temporally,spatially, in ranking, or in any other manner

The drawings and the forgoing description give examples of embodiments.Those skilled in the art will appreciate that one or more of thedescribed elements may well be combined into a single functionalelement. Alternatively, certain elements may be split into multiplefunctional elements. Elements from one embodiment may be added toanother embodiment. For example, orders of processes described hereinmay be changed and are not limited to the manner described herein.Moreover, the actions of any flow diagram need not be implemented in theorder shown; nor do all of the acts necessarily need to be performed.Also, those acts that are not dependent on other acts may be performedin parallel with the other acts. The scope of embodiments is by no meanslimited by these specific examples. Numerous variations, whetherexplicitly given in the specification or not, such as differences instructure, dimension, and use of material, are possible. The scope ofembodiments is at least as broad as given by the following claims.

Embodiments may be provided, for example, as a computer program productwhich may include one or more transitory or non-transitorymachine-readable storage media having stored thereon machine-executableinstructions that, when executed by one or more machines such as acomputer, network of computers, or other electronic devices, may resultin the one or more machines carrying out operations in accordance withembodiments described herein. A machine-readable medium may include, butis not limited to, floppy diskettes, optical disks, CD-ROMs (CompactDisc-Read Only Memories), and magneto-optical disks, ROMs, RAMs, EPROMs(Erasable Programmable Read Only Memories), EEPROMs (ElectricallyErasable Programmable Read Only Memories), magnetic or optical cards,flash memory, or other type of media/machine-readable medium suitablefor storing machine-executable instructions.

FIG. 6 illustrates an embodiment of a computing environment 600 capableof supporting the operations discussed above. The modules and systemscan be implemented in a variety of different hardware architectures andform factors including that shown in FIG. 5.

The Command Execution Module 601 includes a central processing unit tocache and execute commands and to distribute tasks among the othermodules and systems shown. It may include an instruction stack, a cachememory to store intermediate and final results, and mass memory to storeapplications and operating systems. The Command Execution Module mayalso serve as a central coordination and task allocation unit for thesystem.

The Screen Rendering Module 621 draws objects on the one or moremultiple screens for the user to see. It can be adapted to receive thedata from the Virtual Object Behavior Module 604, described below, andto render the virtual object and any other objects and forces on theappropriate screen or screens. Thus, the data from the Virtual ObjectBehavior Module would determine the position and dynamics of the virtualobject and associated gestures, forces and objects, for example, and theScreen Rendering Module would depict the virtual object and associatedobjects and environment on a screen, accordingly. The Screen RenderingModule could further be adapted to receive data from the Adjacent ScreenPerspective Module 607, described below, to either depict a targetlanding area for the virtual object if the virtual object could be movedto the display of the device with which the Adjacent Screen PerspectiveModule is associated. Thus, for example, if the virtual object is beingmoved from a main screen to an auxiliary screen, the Adjacent ScreenPerspective Module 2 could send data to the Screen Rendering Module tosuggest, for example in shadow form, one or more target landing areasfor the virtual object on that track to a user's hand movements or eyemovements.

The Object and Gesture Recognition Module 622 may be adapted torecognize and track hand and arm gestures of a user. Such a module maybe used to recognize hands, fingers, finger gestures, hand movements anda location of hands relative to displays. For example, the Object andGesture Recognition Module could for example determine that a user madea body part gesture to drop or throw a virtual object onto one or theother of the multiple screens, or that the user made a body part gestureto move the virtual object to a bezel of one or the other of themultiple screens. The Object and Gesture Recognition System may becoupled to a camera or camera array, a microphone or microphone array, atouch screen or touch surface, or a pointing device, or some combinationof these items, to detect gestures and commands from the user.

The touch screen or touch surface of the Object and Gesture RecognitionSystem may include a touch screen sensor. Data from the sensor may befed to hardware, software, firmware or a combination of the same to mapthe touch gesture of a user's hand on the screen or surface to acorresponding dynamic behavior of a virtual object. The sensor date maybe used to momentum and inertia factors to allow a variety of momentumbehavior for a virtual object based on input from the user's hand, suchas a swipe rate of a user's finger relative to the screen. Pinchinggestures may be interpreted as a command to lift a virtual object fromthe display screen, or to begin generating a virtual binding associatedwith the virtual object or to zoom in or out on a display. Similarcommands may be generated by the Object and Gesture Recognition Systemusing one or more cameras without the benefit of a touch surface.

The Direction of Attention Module 623 may be equipped with cameras orother sensors to track the position or orientation of a user's face orhands. When a gesture or voice command is issued, the system candetermine the appropriate screen for the gesture. In one example, acamera is mounted near each display to detect whether the user is facingthat display. If so, then the direction of attention module informationis provided to the Object and Gesture Recognition Module 622 to ensurethat the gestures or commands are associated with the appropriatelibrary for the active display. Similarly, if the user is looking awayfrom all of the screens, then commands can be ignored.

The Device Proximity Detection Module 625 can use proximity sensors,compasses, GPS (global positioning system) receivers, personal areanetwork radios, and other types of sensors, together with triangulationand other techniques to determine the proximity of other devices. Once anearby device is detected, it can be registered to the system and itstype can be determined as an input device or a display device or both.For an input device, received data may then be applied to the ObjectGesture and Recognition Module 622. For a display device, it may beconsidered by the Adjacent Screen Perspective Module 607.

The Virtual Object Behavior Module 604 is adapted to receive input fromthe Object Velocity and Direction Module, and to apply such input to avirtual object being shown in the display. Thus, for example, the Objectand Gesture Recognition System would interpret a user gesture and bymapping the captured movements of a user's hand to recognized movements,the Virtual Object Tracker Module would associate the virtual object'sposition and movements to the movements as recognized by Object andGesture Recognition System, the Object and Velocity and Direction Modulewould capture the dynamics of the virtual object's movements, and theVirtual Object Behavior Module would receive the input from the Objectand Velocity and Direction Module to generate data that would direct themovements of the virtual object to correspond to the input from theObject and Velocity and Direction Module.

The Virtual Object Tracker Module 606 on the other hand may be adaptedto track where a virtual object should be located in three-dimensionalspace in a vicinity of a display, and which body part of the user isholding the virtual object, based on input from the Object and GestureRecognition Module. The Virtual Object Tracker Module 606 may forexample track a virtual object as it moves across and between screensand track which body part of the user is holding that virtual object.Tracking the body part that is holding the virtual object allows acontinuous awareness of the body part's air movements, and thus aneventual awareness as to whether the virtual object has been releasedonto one or more screens.

The Gesture to View and Screen Synchronization Module 608, receives theselection of the view and screen or both from the Direction of AttentionModule 623 and, in some cases, voice commands to determine which view isthe active view and which screen is the active screen. It then causesthe relevant gesture library to be loaded for the Object and GestureRecognition Module 622. Various views of an application on one or morescreens can be associated with alternative gesture libraries or a set ofgesture templates for a given view. As an example, in FIG. 1A apinch-release gesture launches a torpedo, but in FIG. 1B, the samegesture launches a depth charge.

The Adjacent Screen Perspective Module 607, which may include or becoupled to the Device Proximity Detection Module 625, may be adapted todetermine an angle and position of one display relative to anotherdisplay. A projected display includes, for example, an image projectedonto a wall or screen. The ability to detect a proximity of a nearbyscreen and a corresponding angle or orientation of a display projectedtherefrom may for example be accomplished with either an infraredemitter and receiver, or electromagnetic or photo-detection sensingcapability. For technologies that allow projected displays with touchinput, the incoming video can be analyzed to determine the position of aprojected display and to correct for the distortion caused by displayingat an angle. An accelerometer, magnetometer, compass, or camera can beused to determine the angle at which a device is being held whileinfrared emitters and cameras could allow the orientation of the screendevice to be determined in relation to the sensors on an adjacentdevice. The Adjacent Screen Perspective Module 607 may, in this way,determine coordinates of an adjacent screen relative to its own screencoordinates. Thus, the Adjacent Screen Perspective Module may determinewhich devices are in proximity to each other, and further potentialtargets for moving one or more virtual object's across screens. TheAdjacent Screen Perspective Module may further allow the position of thescreens to be correlated to a model of three-dimensional spacerepresenting all of the existing objects and virtual objects.

The Object and Velocity and Direction Module 603 may be adapted toestimate the dynamics of a virtual object being moved, such as itstrajectory, velocity (whether linear or angular), momentum (whetherlinear or angular), etc. by receiving input from the Virtual ObjectTracker Module. The Object and Velocity and Direction Module may furtherbe adapted to estimate dynamics of any physics forces, by for exampleestimating the acceleration, deflection, degree of stretching of avirtual binding, etc. and the dynamic behavior of a virtual object oncereleased by a user's body part. The Object and Velocity and DirectionModule may also use image motion, size and angle changes to estimate thevelocity of objects, such as the velocity of hands and fingers

The Momentum and Inertia Module 602 can use image motion, image size,and angle changes of objects in the image plane or in athree-dimensional space to estimate the velocity and direction ofobjects in the space or on a display. The Momentum and Inertia Module iscoupled to the Object and Gesture Recognition Module 622 to estimate thevelocity of gestures performed by hands, fingers, and other body partsand then to apply those estimates to determine momentum and velocitiesto virtual objects that are to be affected by the gesture.

The 3D Image Interaction and Effects Module 605 tracks user interactionwith 3D images that appear to extend out of one or more screens. Theinfluence of objects in the z-axis (towards and away from the plane ofthe screen) can be calculated together with the relative influence ofthese objects upon each other. For example, an object thrown by a usergesture can be influenced by 3D objects in the foreground before thevirtual object arrives at the plane of the screen. These objects maychange the direction or velocity of the projectile or destroy itentirely. The object can be rendered by the 3D Image Interaction andEffects Module in the foreground on one or more of the displays. Asillustrated, various components, such as components 601, 602, 603, 604,605. 606, 607, and 608 are connected via an interconnect or a bus, suchas bus 609.

The following clauses and/or examples pertain to further embodiments orexamples. Specifics in the examples may be used anywhere in one or moreembodiments. The various features of the different embodiments orexamples may be variously combined with some features included andothers excluded to suit a variety of different applications. Examplesmay include subject matter such as a method, means for performing actsof the method, at least one machine-readable medium includinginstructions that, when performed by a machine cause the machine toperforms acts of the method, or of an apparatus or system forfacilitating hybrid communication according to embodiments and examplesdescribed herein.

Some embodiments pertain to Example 1 that includes an apparatus tofacilitate interest-based enhancement in media quality, the apparatuscomprising: detection/reception logic to receive eye trackinginformation from a wearable computing device, wherein the eye trackinginformation contains data relating to pupils of eyes of a user havingaccess to the wearable computing device; and encoding/enhancement logicto enhance quality of at least contents of media, if a size of a pupilis determined to have been altered due to the user's interest in thecontents, wherein the media having the enhanced contents is encoded andcommunicated back to the wearable computing device to be rendered at thewearable computing device.

Example 2 includes the subject matter of Example 1, further comprisingevaluation logic to determine whether the size of the pupil is altereddue to the user having interest in contents of media being rendered atthe wearable computing device, wherein the media includes at least oneof one or more images and one or more videos.

Example 3 includes the subject matter of Examples 1-2, wherein whetherthe size of the pupil is altered due to the user's interest or otherfactors is determined based on a psychophysical state of the user.

Example 4 includes the subject matter of Examples 1-3, wherein thepsychophysical state is obtained from evaluating one or more of displaybrightness, visual context, aural context, biometrics, and deviceinteraction context, wherein the other factors include suddenbrightness, change in weather, and touch of humans or pets.

Example 5 includes the subject matter of Examples 1-4, wherein theevaluation logic is further to create a region of interest (ROI) thatencompassed the contents, wherein the enhanced media contains at leastthe ROI.

Example 6 includes the subject matter of Examples 1-5, wherein thewearable computing device comprises a head-mounted display to performtracking of the eyes to obtain the eye tracking information, wherein thewearable computing device is further to decode and render the mediahaving the enhanced contents.

Example 7 includes the subject matter of Examples 1-6, furthercomprising communication/compatibility logic to facilitate communicationwith the wearable computing device over one or more networks includingone or more of a cloud network, a proximity network, and the Internet.

Some embodiments pertain to Example 8 that includes a method tofacilitating interest-based enhancement in media quality, the methodcomprising: receiving eye tracking information from a wearable computingdevice, wherein the eye tracking information contains data relating topupils of eyes of a user having access to the wearable computing device;and enhancing quality of at least contents of media, if a size of apupil is determined to have been altered due to the user's interest inthe contents, wherein the media having the enhanced contents is encodedand communicated back to the wearable computing device to be rendered atthe wearable computing device.

Example 9 includes the subject matter of Example 8, further comprisingdetermining whether the size of the pupil is altered due to the userhaving interest in contents of media being rendered at the wearablecomputing device, wherein the media includes at least one of one or moreimages and one or more videos.

Example 10 includes the subject matter of Examples 8-9, wherein whetherthe size of the pupil is altered due to the user's interest or otherfactors is determined based on a psychophysical state of the user.

Example 11 includes the subject matter of Examples 8-10, wherein thepsychophysical state is obtained from evaluating one or more of displaybrightness, visual context, aural context, biometrics, and deviceinteraction context, wherein the other factors include suddenbrightness, change in weather, and touch of humans or pets.

Example 12 includes the subject matter of Examples 8-11, furthercomprising creating a region of interest (ROI) that encompassed thecontents, wherein the enhanced media contains at least the ROI.

Example 13 includes the subject matter of Examples 8-12, wherein thewearable computing device comprises a head-mounted display to performtracking of the eyes to obtain the eye tracking information, wherein thewearable computing device is further to decode and render the mediahaving the enhanced contents.

Example 14 includes the subject matter of Examples 8-13, furthercomprising facilitating communication with the wearable computing deviceover one or more networks including one or more of a cloud network, aproximity network, and the Internet.

Some embodiments pertain to Example 15 that includes a graphicsprocessing system comprising a computing device having memory coupled toa processor, the processor to: receive eye tracking information from awearable computing device, wherein the eye tracking information containsdata relating to pupils of eyes of a user having access to the wearablecomputing device; and enhance quality of at least contents of media, ifa size of a pupil is determined to have been altered due to the user'sinterest in the contents, wherein the media having the enhanced contentsis encoded and communicated back to the wearable computing device to berendered at the wearable computing device.

Example 16 includes the subject matter of Example 15, wherein theoperations further comprise determining whether the size of the pupil isaltered due to the user having interest in contents of media beingrendered at the wearable computing device, wherein the media includes atleast one of one or more images and one or more videos.

Example 17 includes the subject matter of Examples 15-16, whereinwhether the size of the pupil is altered due to the user's interest orother factors is determined based on a psychophysical state of the user.

Example 18 includes the subject matter of Examples 15-17, wherein thepsychophysical state is obtained from evaluating one or more of displaybrightness, visual context, aural context, biometrics, and deviceinteraction context, wherein the other factors include suddenbrightness, change in weather, and touch of humans or pets.

Example 19 includes the subject matter of Examples 15-18, wherein theoperations further comprise creating a region of interest (ROI) thatencompassed the contents, wherein the enhanced media contains at leastthe ROI.

Example 20 includes the subject matter of Examples 15-19, wherein thewearable computing device comprises a head-mounted display to performtracking of the eyes to obtain the eye tracking information, wherein thewearable computing device is further to decode and render the mediahaving the enhanced contents.

Example 21 includes the subject matter of Examples 15-20, wherein theprocessor comprises a graphics processor, wherein the graphics processoris co-located with an application processor on a common semiconductorpackage.

Example 22 includes at least one non-transitory or tangiblemachine-readable medium comprising a plurality of instructions, whenexecuted on a computing device, to implement or perform a method asclaimed in any of claims or examples 8-14.

Example 23 includes at least one machine-readable medium comprising aplurality of instructions, when executed on a computing device, toimplement or perform a method as claimed in any of claims or examples8-14.

Example 24 includes a system comprising a mechanism to implement orperform a method as claimed in any of claims or examples 8-14.

Example 25 includes an apparatus comprising means for performing amethod as claimed in any of claims or examples 8-14.

Example 26 includes a computing device arranged to implement or performa method as claimed in any of claims or examples 8-14.

Example 27 includes a communications device arranged to implement orperform a method as claimed in any of claims or examples 8-14.

Example 28 includes at least one machine-readable medium comprising aplurality of instructions, when executed on a computing device, toimplement or perform a method or realize an apparatus as claimed in anypreceding claims.

Example 29 includes at least one non-transitory or tangiblemachine-readable medium comprising a plurality of instructions, whenexecuted on a computing device, to implement or perform a method orrealize an apparatus as claimed in any preceding claims.

Example 30 includes a system comprising a mechanism to implement orperform a method or realize an apparatus as claimed in any precedingclaims.

Example 31 includes an apparatus comprising means to perform a method asclaimed in any preceding claims.

Example 32 includes a computing device arranged to implement or performa method or realize an apparatus as claimed in any preceding claims.

Example 33 includes a communications device arranged to implement orperform a method or realize an apparatus as claimed in any precedingclaims.

The drawings and the forgoing description give examples of embodiments.Those skilled in the art will appreciate that one or more of thedescribed elements may well be combined into a single functionalelement. Alternatively, certain elements may be split into multiplefunctional elements. Elements from one embodiment may be added toanother embodiment. For example, orders of processes described hereinmay be changed and are not limited to the manner described herein.Moreover, the actions of any flow diagram need not be implemented in theorder shown; nor do all of the acts necessarily need to be performed.Also, those acts that are not dependent on other acts may be performedin parallel with the other acts. The scope of embodiments is by no meanslimited by these specific examples. Numerous variations, whetherexplicitly given in the specification or not, such as differences instructure, dimension, and use of material, are possible. The scope ofembodiments is at least as broad as given by the following claims.

What is claimed is:
 1. An apparatus comprising: one or processors to:receive eye tracking information from a wearable computing device,wherein the eye tracking information contains data relating to pupils ofeyes of a user; and enhance quality of contents of media when a size ofa pupil of an eye is determined to have been altered due to the user'sinterest in the contents, the media having the enhanced contents isencoded and communicated back to the wearable computing device to berendered at the wearable computing device.
 2. The apparatus of claim 1,wherein the one or more processors are further to determine whether thesize of the pupil is altered due to the user having interest in contentsof media being rendered at the wearable computing device, wherein themedia includes at least one of one or more images and one or morevideos.
 3. The apparatus of claim 2, wherein whether the size of thepupil is altered due to the user's interest or other factors isdetermined based on a psychophysical state of the user.
 4. The apparatusof claim 3, wherein the psychophysical state is obtained from evaluatingone or more of display brightness, visual context, aural context,biometrics, and device interaction context, wherein the other factorsinclude sudden brightness, change in weather, and touch of humans orpets.
 5. The apparatus of claim 1, wherein one or more processors arefurther to create a region of interest (ROI) that encompassed thecontents, wherein the enhanced media contains at least the ROI.
 6. Theapparatus of claim 1, wherein the wearable computing device comprises ahead-mounted display to perform tracking of the eyes to obtain the eyetracking information, wherein the wearable computing device is furtherto decode and render the media having the enhanced contents.
 7. Theapparatus of claim 1, one or more processors are further to facilitatecommunication with the wearable computing device over one or morenetworks including one or more of a cloud network, a proximity network,and the Internet.
 8. A method comprising: receiving eye trackinginformation from a wearable computing device, wherein the eye trackinginformation contains data relating to pupils of eyes of a user; andenhancing quality of at least contents of media, if a size of a pupil isdetermined to have been altered due to the user's interest in thecontents, wherein the media having the enhanced contents is encoded andcommunicated back to the wearable computing device to be rendered at thewearable computing device.
 9. The method of claim 8, further comprisingdetermining whether the size of the pupil is altered due to the userhaving interest in contents of media being rendered at the wearablecomputing device, wherein the media includes at least one of one or moreimages and one or more videos.
 10. The method of claim 9, whereinwhether the size of the pupil is altered due to the user's interest orother factors is determined based on a psychophysical state of the user.11. The method of claim 10, wherein the psychophysical state is obtainedfrom evaluating one or more of display brightness, visual context, auralcontext, biometrics, and device interaction context, wherein the otherfactors include sudden brightness, change in weather, and touch ofhumans or pets.
 12. The method of claim 8, further comprising creating aregion of interest (ROI) that encompassed the contents, wherein theenhanced media contains at least the ROI.
 13. The method of claim 8,wherein the wearable computing device comprises a head-mounted displayto perform tracking of the eyes to obtain the eye tracking information,wherein the wearable computing device is further to decode and renderthe media having the enhanced contents.
 14. The method of claim 8,further comprising facilitating communication with the wearablecomputing device over one or more networks including one or more of acloud network, a proximity network, and the Internet.
 15. At least onemachine-readable medium comprising instructions which, when executed bya computing device, cause the computing device to facilitate operationscomprising: receiving eye tracking information from a wearable computingdevice, wherein the eye tracking information contains data relating topupils of eyes of a user; and enhancing quality of at least contents ofmedia, if a size of a pupil is determined to have been altered due tothe user's interest in the contents, wherein the media having theenhanced contents is encoded and communicated back to the wearablecomputing device to be rendered at the wearable computing device. 16.The machine-readable medium of claim 15, wherein the operations furthercomprise determining whether the size of the pupil is altered due to theuser having interest in contents of media being rendered at the wearablecomputing device, wherein the media includes at least one of one or moreimages and one or more videos.
 17. The machine-readable medium of claim15, wherein whether the size of the pupil is altered due to the user'sinterest or other factors is determined based on a psychophysical stateof the user.
 18. The machine-readable medium of claim 17, wherein thepsychophysical state is obtained from evaluating one or more of displaybrightness, visual context, aural context, biometrics, and deviceinteraction context, wherein the other factors include suddenbrightness, change in weather, and touch of humans or pets.
 19. Themachine-readable medium of claim 15, wherein the operations furthercomprise creating a region of interest (ROI) that encompassed thecontents, wherein the enhanced media contains at least the ROI.
 20. Themachine-readable medium of claim 15, wherein the wearable computingdevice comprises a head-mounted display to perform tracking of the eyesto obtain the eye tracking information, wherein the wearable computingdevice is further to decode and render the media having the enhancedcontents.